The present invention is directed to a system for support of packet transmission in processor-based networks using serial interconnects. In particular, the system of the invention provides dynamic ordering support for packets in response to errors in such networks, in particular in ringlet topologies.
Serial interconnects in computer systems are subject to a number of different types of service interruptions. For example, when a node on a network encounters a CRC (cyclical redundancy check) error in a packet, that packet cannot be accepted. The node that sent the packet learns, generally indirectly (such as by a timeout or through the use of idle packets) of the error, and eventually must resend the packet.
Resending the packet may not be a simple matter, especially if the network implements an ordering scheme, such as relaxed memory ordering (RMO), strong sequential ordering (SSO), or orderings of other or intermediate stringency. In a packet-switched network with such an ordering scheme, packets preceding and following a packet giving rise to a CRC error may need to be resent by the producer node to the target node.
A particular problem arises when such a packet contains a nonidempotent command, i.e. a command which, once it is executed at the target node, changes the state of that node, such that reexecution of the command at that node would yield different results from the first execution; in this case, if the command is resent to the node and executed again, undesired or unforeseen results are likely to take place.
Thus, a system is needed wherein errors in packets can be accommodated by resending the packets to the target node, while maintaining support for idempotent commands. In particular, such a system is needed that also supports various levels of ordering schemes.
A system is presented which provides for resending of packets that have resulted in CRC errors, by maintaining a state at each receive node of all known good packets. When packets need to be resent, the local nodes know whether they have processed the resent packets before, and know which the last known good packet was, and in this way are able to avoid reprocessing already executed commands, including nonidempotent commands. The producer node that has detected an error, e.g. a CRC error, causes the resend loop to commence, and backs up the sending of packets to the last packet where a valid acknowledgment was received, and then resends all packets since that last valid-ack packet. If all nodes return valid acks from that packet, then the system has properly been restored, and proceeds with additional packets. If not, then the producer node backs up again to that last valid-ack packet, and begins again, and repeats until its received valid acks are in synch with its sent packets. This strictly preserves the ordering of the packets.
This application relates specifically to the treatment of such error conditions in a ringlet network, while maintaining SSO support and support for nonidempotent commands. A more sophisticated system, which uses mechanisms such as those described here in addition to logic and methods for accommodating busy retries at the same time as handling error conditions, is described in applicant""s patent application Ser. No. 08/673,850 filed Jul. 1, 1996 now U.S. Pat. No. 5,864,677, entitled System for Preserving Sequential Ordering and Supporting Idempotent Commands in a Ring Network with Busy Nodes by van Loo et al. A yet more complex system which additionally deals with overloaded or failed nodes on a ringlet network is described in applicant""s patent application Ser. No. 08/673,849 filed Jul. 1, 1996, now U.S. Pat. No. 6,065,052, entitled System for Maintaining Strongly Sequentially Ordered Packet Flow in a Ring Network System with Busy and Failed Nodes by van Loo et al. Those two patent applications are incorporated herein by reference.